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ABSTRACT 



Score changes on admissions tests such as the Scholastic Aptitude Test (SAT) can vary 
widely among individuals repeating the test. To a large extent these score changes 
probably reflect the influence of errors of measurement as test candidates with low 
initial scores usually experience score gains upon retesting while test candidates with 
high initial scores often show score losses. Besides this phenomenon of scores re- 
gressing toward the mean upon test repetition, student self-selection may afreet score 
change as test candidates who perceive their initial scores as underestimates of their 
true abilities decide to repeat the test. This study applies a procedure which yields 
estimates of true score change on the SAT adjusted for regression effects and student 
self-selection. It is shown that student self-selection in deciding to repeat an ad- 
missions test probably involves factors in addition to the measurement error attributable 
to variations in aspects of test specifications and to variations in responses of test 
candidates across forms, and that estimated true score change remains nearly constant 
across initial score levels in contrast to the negative slope of observed score change 
across initial score levels. 
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INTRODUCTION 



Each year several hundred thousand students who had taken the Scholastic Aptitude Test 
(SAT) as high school juniors elect to take the test again as seniors. The distribution 
of these junior-senior score changes, typically with a mean near 15 points and a stand- 
ard deviation in the neighborhood of 50 points (Donlon and Angoff, 1971; Educational 
Testing Service, 1979), includes fairly frequent gains in excess of 65 points as well 
as regular losses beyond 35 points on the SAT's 200-800 point scale. One-third of the 
students who take the SAT as juniors and again as seniors will experience score changes 
greater than 50 points. For every hundred students repeating the test, five students 
will gain at least 100 points and one student will lose at least 100 points (Educational 
Testing Service, 1979). 

The widespread of such score changes on an admissions test invites misinterpreta- 
tion. Test candidates unfamiliar with the technical aspects of measurement will find 
it difficult to understand dramatic score changes in terms of inherent error. It is far 
simpler for students, parents and counselors to accept all score increases as a reflec- 
tion of real growth in academic ability or to attribute them to the effectiveness of 
some intervening event (e.g., a particular course or instructor, a specific program of 
test preparation). All score decreases, on the other hand, may be viewed as a sign of 
procedural error on the part of the test publisher or as evidence of further decline in 
the quality of education. Thus, gains among selected students have been interpreted as 
indicative of the impact of special test preparation while losses among other selected 
students have prompted considerable concern about possible implications for school 
practices (e.g., Kendrick, 1967; Messick, 1980). The extent to which these score 
changes represent the consequences of measurement error seems unclear but certainly 
bears on the interpretation of score differences. 



Regression Effects and Student Self-Selection 

The fallibility of the test as a measurement instrument usually accounts for most of the 
score difference observed between administrations of a test, as Duggan (1959) suggests is 
the case for score changes on the SAT. The unreliability and dispersion of SAT score 
changes follow from the reliability of the test, the test-retest correlation, and the 
test's standard deviation according to standard formulas such as those given by Lord 
(1963), McNemar (1958), and Overall and Woodward (197 5). For students taking the test ' 
in May 1979 as juniors and again in November 1979 as seniors, it can be shown that the 
reliability of differences in SAT-Verbal scores is only .25 and of differences in SAT- 
Mathematical scores only .23 (see Alderman, 1981 for estimates of test reliability among 
repeaters and Tables 1 and 2 below for standard deviations and test-retest correlations). 
Similarly, the standard deviations of these difference scores are 48 points for the SAT-V 
and 53 points for the SAT-M. Since there will be approximately 32 percent of the cases 
at least one standard deviation away from the mean and 5 percent at least two standard 
deviations away from the mean in any normal distribution, the frequency of SAT score 
changes greater than 50 points and greater than 100 points conforms to expectations for 
the test. And the degree of importance attached to the observed score changes for an 
individual or a selected group of individuals should be consistent with the low reli- 
ability of these score changes. 




^ , , „ - - fc — among 

students near the mean of the distribution of scores. At the extremes of the score scale, 
however, the test is less reliable as errors of measurement have probably served to de- 
press low scores and to raise high scores to some extent. Consequently, an examinee 
whose initial score was either very low or very high may experience a score change far 
greater than the standard error of measurement upon repeating the test. Because the SAT 
has a standard error of measurement usually near 30 points (e.g., Educational Testing 
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Service, 1980) and a standard deviation for difference scores close to 50 points, the 
majority of test candidates repeating the SAT will actually experience score changes, 
gains and losses, greater than the standard error of measurement. Sizeable score changes 
are most apparent among students with extreme low scores on an initial administration of 
the SAT as real growth in ability between test administrations acts to exaggerate further 
score increments due to regression. For extreme high scores, real growth act& in the 
opposite direction as it diminishes the magnitude of score decrements due to regression. 

The interaction of regression toward the mean upon repetition of a test and real 
growth between test administrations in the trait reflected by scores on a test can lead 
to unusual and sometimes counter-intuitive contrasts between observed score changes 
found through simple differences and true score changes free of the influences of errors 
of measurement. For test candidates with high initial scores the observed score change 
is apt to be negative while the true score change is probably positive. Also, an observed 
loss at one score level may reflect greater growth in ability than an observed gain at 
another score level as when an individual with a high initial score loses points and an 
individual with a low initial score gains points on a subsequent test. Empirical evi- 
dence of scores regressing toward the mean on multiple administrations of an admissions 
test appears in Rock and Werts (1980), and a thorough discussion of possible theoretical 
patterns of score change due to regression across multiple test administrations appears 
in Nesselroade, Stigler, and Baltes (1980). 

Besides regression effects as a component of score change upon repetition of a test, 
there may be student self-selection in deciding to repeat a test prompted by negative 
errors of measurement on the initial test administration (e.g., Jacobs, 1966). Alderman 
(1981) demonstrates that the test scores expected on the basis of student performance on 
separate equating sections and student background variables such as high school rank and 
years of mathematics study overpredict the Initial verbal and mathematical scores ob- 
tained by students who later repeat the SAT. Students electing to repeat an admissions 
test apparently do so in part because they perceive their initial scores as underestimates 
of their true abilities. This implies a nonzero, negative sum of errors of measurement on 
repeaters' initial test scores which would make the observed difference in mean scores 
between test administrations an overestimate of true score change (e.g., Lord, 1956) and 
preclude the application of existing models for measuring change (e.g., Lord, 1963). 

The purpose of this study is to estimate score change while correcting for regression 
effects and student self-selection. The method adopted for estimating score change among 
students taking the SAT as juniors and repeating the test as seniors depends on a con- 
current verbal or mathematical score available through shorter but otherwise parallel 
equating sections. Each administration of the SAT includes a separate experimental sec- 
tion for equating test forms or for pretesting items which does not enter into the oper- 
ational scores reported back to test candidates and thus should not be a factor in student 
self-selection in deciding to repeat the test. Using scores on equating sections and 
adjusting for regression toward the mean should lead to estimates of score change more 
stable across initial score levels rather than a pronounced negative correlation between 
observed score change and initial score (e.g., Linn and Slinde, 1977). It should also 
lead to a lower estimate of mean score change if negative errors of measurement on an 
initial test administration prompt students to repeat a test. Such findings would in- 
dicate the influence of measurement error on observed score change and clarify the ex- 

of real score change at various levels of initial test scores. 
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Anai^ a of Score Change 

The procedure employed here in estimating sc^o change follows a procedure described by 
F.M. Lord (personal communication, August 1^ 1980). It depends on accessing and ex- 
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pressing scores for equating sections of the SAT on the same scale as the regular verbal 
or mathematical scores reported to test candidates. Let X denote a regular SAT-V or 
SAT-M score and Y a score on a corresponding verbal or mathematical equating section. 
All X scores on different forms are equated on a common scale (i.e., the 200-800 point 
scale for the SAT), and Y scores on different sections can also be expressed on a common 
scale. Further assume that there is a linear relationship between true scores on X and 
Y. Score X contains a true score, T, and an error of measurement, E: 

X = T + E. 

Score Y similarly contains a true score, aT + b where a and b are scaling constants and 
an error, F: 

Y - aT + b + F. 

After taking X { and Y { for the first time as juniors, a subgroup of examinees decides to 
retake X 2 and Y 2 as seniors. Another random sample of examinees has taken X Q and Y Q , here 
chosen from the same initial test administration in the junior year for convenience It 
is assumed that all errors are unbiased with an expected value of zero except E^ because 

the decision to retake the SAT was partially based on score X. . Averaging across examinees 
it will be found that: 1 

*o = V Y o " a? o + b: 

Xj = T, + B 1§ Y, = aT, + b; 
X 2 = T 2 , Y 2 = aT 2 + b. 

From this set of relationships we can determine the scaling constants as well as T, : 

a - (Y 2 - y/(X 2 - X Q ), 

The estimate of average true score change among all candidates repeating the test as 
Xj and X 2 is then: 

This estimate of average true score change from examinee performance on equating sections 
and scaling constants can be contrasted with the average observed score change, x 2 " X l # 
If the negative errors of measurement on initial test scores encouraged some students to 
retake the test, it should be the case that is nonzero and negative, 

Ej - X t - Tj with X r - T 1 < 0, 

and that 

X 2 -X, >T 2 -T r 
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For individuals an estimate of true score change based on their performance on 
equating sections would be: 

T - T =z ^ (Y - Y t ) + f l - F 2 . 

Y 2" Y 0 • 

This T 2 - may be averaged over individuals at the same observed initial score level, 
in order to show how T 2 - T { varies with X { , since Fj - F 2 should average to zero. These 
estimates of T 2 - at different initial score levels should result in a slope along Xj 
less steep than that for X 2 - X { along X { as the latter observed score difference will be 
subject to regression effects while the former estimated score difference should be in- 
dependent of errors of measurement on the initial test. 



Samples of Test Candidates 

Samples of test records representative of the most common pattern of test repetition among 
secondary school students were drawn from, data files. These records came from the admin- 
istrations of the SAT given in May 1979 and November 1979, and included only students who 
had taken the SAT for the first time in May 1979 as juniors and repeated the test in 
November 1979 as seniors without any intervening administrations of the test. There were 
81,959 such students, but the analysis of score change also required scores on verbal or 
mathematical equating sections for both test administrations. Of the ten experimental 
sections spiraled in the administration of the SAT given in May 1979, there were two 
verbal and two mathematical equating sections. Of the ten experimental sections spiraled 
in the administration of the SAT given in November 1979, there were three verbal and three 
mathematical equating sections. Scores on Experimental sections were available for all 
31,971 students who had taken either a verbal or mathematical equating section in May 1979 
and fit the May-November pattern for test repetition. It was possible to retrieve only 
129,878 records with scores on experimental! sections from the test administration given 
in November 1979 and just 9,148 records coiAd be matched against the 31,971 cases from the 
test administration of May 1979. A total of 5,277 examinees had taken equating sections 
in both test administrations and were matched through these data files from May 1979 and 
from November 1979. Of these there were 1,325 examinees with scores on verbal equating 
sections for both test administrations (Y^ and Y 2 on Verbal equating sections corresponding 

to and X 2 on regular verbal sections) and 1,312 examinees with scores on mathematical 

equating sections for both test administrations (Yj and Y 2 on mathematical equating sections 

corresponding to X^ and X 2 on regular mathematical sections). 

It was also necessary to convert scores on equating sections to a common scale as a 
prerequisite to the proposed analysis of score change. This could be accomplished in a 
straightforward manner within each test administration since the spiraling of experimental 
sections among test candidates should result in comparable groups of examinees taking each 
section and thereby permit the linking of scores based on means and standard deviations. 
A verbal equating section from the test administration given in May 1979 and another from 
the test administration given in November 1979 were among the experimental sections 
spiraled in the administration of the SAT in November 1980, which established a link 
between the equating sections ftom May 1979 and from November 1979. The same spiraling 
occurred in November 1980 for a mathematical equating section from the test administration 
given in May 1979 and for another mathematical equating Section from the test administra- 
tion given in November 1979. Spiraling of experimental sections made it possible to link 
equating scores within and between test administrations. 

Because the data files with results on experimental sections from the test adminis- 
tration given in November 1980 were incomplete and because the matches of these data files 
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from May and November 1979 depended on linking equating scores within and between test 
administrations, a second sample of the same pattern of test repetition was drawn using 
complete data files from the test administration of May 1979 and random samples of equating 
sections kept from the test administration of November 1979. This sample was restricted 
to 1,020 examinees who had taken the particular verbal or mathematical equating sections 
rfhich were included in the test administration of May 1979 and November 1979 and were 
subsequently spiraled together in the Lest administration of November 1980. Among these 
students there were 152 examinees with the respective pair of verbal equating sections 
lYj and Y 2 on verbal equating sections corresponding to Xj and X» on regular verbal 

sections) and 113 examinees with the respective pair of mathematical equating sections 
(Yj and Y 2 on mathematical equating sections corresponding tc Xj and X^ on regular 
mathematical sections) for May 1979 and November 1979. 

Random samples of test candidates who had taken the verbal equating section in May 
1979, which was later also given in November 1980, as well as test candidates who had 
taken the corresponding mathematical equating section were available from data files 
routinely retained for the SAT (X Q and Y Q ) . 



RESULTS AND DISCUSSION 

A summary of examinees 1 scores on reporting and equating sections of the administrations 
of the SAT given in May 1979 and November 1979 appears in Table 1. The random verbal and 
mathematical equating samples from the test administration given in May 1979, 1,800 and 
1,755 test candidates respectively, provide information on test performance, X Q and Y Qf 
necessary for the analysis of score change. The random verbal and mathematical equating 
samples drawn from students taking the SAT as juniors in May 1979, Xj and Y' f and re- 
peating the test as seniors in November 1979, X' and Y', represent a merger ,of particular 
pairs of equating sections based on complete records for test candidates from May 1979 
and on random files of test candidates from November 1979; these samples of 152 and 113 
test candidates with verbal or mathematical equating sections in both test administrations 
depended on only one link between equating sections in forming a common score scale. The 
largest samples of test candidates with results on reporting and equating sections as 
juniors and seniors, 1,325 students with verbal equating sections in both test administra- 
tions (i.e., Y 1 and Y 2 in the verbal equating sample for file matches) and 1,312 students 
with mathematical equating Sections in both test administrations (i.e., Y x and Y 2 in the 
mathematical equating sample for file matches), represent a match of complete data files 
from May 1979 and incomplete data files from November 1979; these samples depended on 
multiple links within and between test administrations in forming a common score scale 
for equating sections. 

The test results in Table 1 show that secondary school juniors electing to repeat the 
SAT as seniors do almost as well on their initial test administration as their peers but 
as seniors do noticeably better on their second test administration than their peers. 
Only two points separate the SAT-Verbal scores and only three points the SAT-Mathematical 
scores of all junior examinees and junior-senior repeaters on the test administration for 
the junior year. But the junior-senior repeaters exceed the performance* of all senior 
examinees by 16 points on SAT-Verbal scores and 22 points on SAT-Mathematical scores in 
the senior year. These latter differences probably reflect as much on idiosyncracies in 
student choices of test administrations and th. ability of the respective groups as on 
the score change possible through practice and growth, especially since junior examinees 
overall attained higher SAT-V and SAT-M scores in May 1979 than did senior examinees from 
the same high school cohort in November 1979. 
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TABLE 1. Means and Standard Deviations of Repprting and Equating Scores Across Test Administrations 



Junior Year (May 1979) 



Group 



SAT-Verbal 
Mean sd 



SAT-Math. 
Mean sd 



Equating Sect. 
Mean sd 



Senior Year (November 1979) 
SAT-Verbal SAT-Math . 



Mean sd 



Mean 



sd 



Equating Sect. 
Mean sd 



All Examinees 253,354 

Junior examinees 223,394 
Verbal equating 

sample (X Q , Y Q ) 1,800 

Mathematical equating 

sample (X Q , Y Q ) 1,755 

All Examinees 348,954 

Senior examinees 32b, 601 

Junior-Senior Repeaters 81,959 



Random sample 
Verbal equating 
sample (Xj, Y'; 

- XJ, Y') 

Mathematical equat- 
ing sample (Xj, Yj; 

X' Y' ) 

File matches 
Verbal equating 
sample (X , Y.; 

V V 

Mathematical equat- 
ing sarcple (Xj, Y^; 

x ? , \ 2 ) 



1,020 
152 

113 
5,277 

1,325 

1,312 



432 
439 



107 
105 



478 
486 



113 
111 



429.76 


107.33 






16.26 


7. 


94 




















481.75 


114.81 


10.25 


6. 


06 






























434 


107 


476 


114 






















434 


106 


477 


114 








437 


97 


483 


104 








450 


99 


498 


105 








435.67 


97.27 


481.36 


102.36 








449. 4* 


98.43 


493.19 


103.41 








427.43 


93.97 


484.41 


107.01 


1' .30 


7. 


29 


447.63 


94.64 


490.86 


111.97 


17 


85 


7.15 


437.52 


99.38 


477.43 


89.91 


10.35 


5. 


17 


452.39 


105.23 


498.85 


98.15 


11 


.38 


5.71 


436.85 


99.89 


480.45 


106.20 








449.55 


99.94 


496.58 


106.31 









439.93 98.24 483.32 104.21 16.87 7.43 
435.99 99.49 478.45 106.58 10.24 5.69 



452.42 98.21 498.56 104.95 18.63 



7.76 



448.32 99.25 495.55 105.79 10.84 5.68 
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TABLE 2. Intercorrelations Among Reporting and Equating Scores in the Junior and 
Senior Year 

~ , — a 

Junior Year Senior Year 

Equating Equating 
SAT-V SAT-M Section SAT-V SAT-M Section 



Junior 


SAT-V 




.63 


.86 


.88 


.62 


.83 


year 


SAT-M 


.61 




.60 


.60 


.87 


.60 




Equating section 


.58 


.85 




.86 


.59 


• 79, 


Senior 


SAT-V - 


.88 


.58 


.56 




.60 


.85 


year 


SAT-M 


.59 


.87 


.83 


.57 




.60 




Equating section 


.55 


.82 


.79 


.54 


.85 





Note: Entries above the diagonal reflect correlations for the 1,325 repeaters with 
a verbal equating section in both their junior and senior year; entries below the diagonal 
reflect correlations for the 1,312 repeaters with a mathematical equating section in both 
their junior and senior year. 



Table 2 presents correlations between reporting and equating scores for students 
taking the SAT as juniors and seniors. The high correlations between SAT-Verbal scores 
across years, r XjX 2 * *^ 9 anc * ^ etween SAT-Mathmatical scores across years, r XjX 2 * 9 
imply a high degree of consistency in test performance from the junior year to the senior 
year. Similar stability is reflected in correlations between corresponding reporting and 
equating sections within years, r XjYj and r X 2 Y 2 " " 85 " Tnese findings would seem to 

weaken the argument that students decide, to repeat the test in part because they perceive 
that negative errors of measurement have depressed their initial test scores. Such 
errors should also lessen the correlation of test scores across occasions unless errors 
of measurement persist across occasions for certain individuals (i.e., p - - > 0) . 

Table 2 also shows that the correlation between SAT-Verbal and SAT-Mathematical scores 
remains low among students repeating the SAT as juniors and seniors, x « .60, in contrast 
to the correlation found among students taking the SAT only once as juniors, r - .73 
(Alderman, 1981). This both strengthens the, possibility that errors of measurement per- 
sist across occasions and suggests an alternative explanation for some students deciding 
to retake the test, as they perhaps believe that a disparity between verbal and mathe- 
mathical scores signif ies an opportunity for improving^ the lower score upon test repe- 
tition rather than relative strengths in the respective abilities. 

Score changes appear in Table 3 as observed differences between test administrations 
in the junior and senior years, X^ - Xj, and estimated diffetences between test administra- 
tions in the junior and senio* years, T 2 ~ T l* alon 8 with the average error of measurement 
on the initial test administration, 45 . For the random equating samples the expected re- 
lationships hold between observed score changes and * estimated score change and tend to 
confirm student self- select ion in deciding to repeat the test on_the basis_of negative 
errors of measurement on the initial test administration (i.e., x 2 ~ X j > T 2 ~ T i and 
Ej < 0) . However, it should also be noted that the largest average verbal score gain 

and the largest average mathematical score gain, as reported above in Table 1» occurred 
in these respective random equating samples. 

The larger samples drawn from matches of data files from the pertinent test admin- 
istrations offer conflicting evidence about the necessity for this procedure for the 
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TABLE 3. Observed and Estimated Scores Changes from Junior to Senior Year 
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Group 



Observed Estimated Estimated 

x Mean Mean Measurement 

Change Change Error 



SAT-Verbal ~ 

Random samples 152 20.20 17.48 

(Xj , Y j \ X,!| 9 

File matches 1,325 12.49 16.84 

(Xj , Yj ; X2 » Y2) 



-2.72 



+4.35 



SAT-Mathematical 
Random Samples 

(Xj, ^1' ^2' ^2^ 
File matches 

(Xj , Y j I X^ » Y^) 



113 



1,312 



21.42 



17.10 



15.53 



14.03 



-5.88 



-3.07 



analysis of score change and about negative errors of measurement on initial scores 
prompting students to repeat a test. Although the observed score change exceeds the 
estimated score change for the SAT-M, the reverse is the. case for the SAT-V, as shown 
in Table 3. Yet a positive error of measurement should lead students to accept their 
initial test scores rather than to repeat the test. Of course this estimated positive 
error of measurement on initial test scores may simply reflect the large standard error 
of measurement inherent to difference scores or the inevitable error arising from the 
complexity of links among equating sections within and between test administrations. 

Although the particular procedure for the analysis of score change followed here 
does yield estimates of average score change closer together than observed score dif- 
ferences for the random samples and the file matches (i.e., 17.48 and 16.84 versus 20.20 
and 12.49 for verbal scores, and 15.53 and 14.03 versus 21.42 and 17.10 for mathematical 
scores), the procedure may not fully compensate for student self-selection in test repe- 
tition. Adjustments for student self-selection based on equating scores alone risk a 
correlation between errors of measurement on equating and reporting sections (i.e., 
Pj^Pj > 0). For example, atypical test performance attributable to a student's physical 

health or emotional stress is apt to affect scores on all sections. Moreover, student 
self-selection in deciding to repeat a test may occur because scores seem inconsistent 
with high school rank, years of mathematics study, years of English study, and other 
variables aside from scores on equating sections (see Alderman, 1981). 

Figures 1 and 2 Illustrate the appropriateness and usefulness of the estimates of 
score change in adjusting for the effects of regression when students repeat tests. The 
points plotted in these figures represent observed and estimated average true score 
change by level of initial test score for the samples formed by matching data files 
(see Appendix A). The solid lines represent the best least-squares linear fit for 
observed score changes, and the broken lines the same for estimated score changes. 
The negative slopes for observed score change by initial test score clearly reflect 
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» Estimated 



FIGURE 2. Observed and Estimated Score Change by Initial Level of SAT-Mathematical Scores. 

Points for estimated score change at initial score levels of 230 and 770 fall 
beyond the bounds of the abscissa* 
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the expected negative correlations indicative of scores regressing towara the mean 
upon retesting, *(X2-Xi)Xi * -« 2 ^ 9 for verbal scores and *(X2-Xi)Xi " *"- 269 for 
mathematical scores. These correlations between observed score change and initial 
test score correspond closely to their respective expected values (i.e., -.232 for 
verbal scores and -.259 for mathematical scores). The slopes of nearly zero for 
estimated true score change by initial test score, r^-T^X^ * ""'020 for verbal 
scores and r(T2-Ti)Xi * ~' 025 for mathematical scores, contrast sharply with those 
for observed score change by initial test score and show that true score change is 
nearly constant acro3s different levels of initial test scores. The extent to which 
measurement error affects score change as initially low scores increase and initially 
high scores decrease is shown by the difference in slopes between observed score change 
in these figures. 

Rescaling scores on equating sections to the same scale as reported for regular 
SAT scores results in estimates of score change seemingly free from the typical re- 
gression effect arising from measurement error. This suggests the independence of 
errors of measurement on equating and reporting sections, an implicit assumption for 
this analysis of score change. Fuither, it would appear to be the case that the errors 
of measurement inherent to SAT-Vert-al and SAT-Mathematical scores stem primarily from 
variation in content, difficulty, and other aspects of the specifications for parallel 
test forms* and from variation in individual responses like guessing, pacing, and 
omitting across test sections. Sources .of measurement error like an individual's 
health or the conditions of test administration would likely affect equating scores 
as well as reporting scores, and thereby lead to parallel slopes for observed and 
estimated score change by initial test score rather than the marked differences in 
slopes apparent in Figures 1 and 2. Chronic poor test performance may prompt some 
students to retake the SAT if such a component of measurement error remains fairly 
constant across test sections and administrations and Is independent of other sources 
of measurement error. The majority of secondary school juniors repeating the test 
probably do so because they already planned test repetition as seniors or perceived 
their scores on the initial test administration as underestimates of their abilities. 

While the use of equating sections or other types of parallel and concurrent test 
forms might yield better estimates of score change across levels of initial test scores, 
such a procedure is dependent upon aggregating data at those score levels and would still 
give an unreliable and unstable estimate of score change for an individual. Errors of 
measurement on equating sections (i.e., Fj and F 2 > are a part of the estimated score 

change for individuals. The standard deviations for the observed and estimated verbal 
score changes depicted as averages in Figure I were 48.86 points and 46.70 points, re- 
spectively. For the observed and estimated mathematical score changes in Figure 2 the 
standard deviations were 54.40 points and 85.57 points, respectively. Although the 
standard deviations for observed and estimated verbal score changes are comparable, the 
standard deviation for estimated mathematical score changes is obviously greater than 
the standard deviation for observed mathematical score changes. This simply reflects 
the greater opportunity for errors of measurement in sampling mathematical ability on 
an equating section which is half the length of the regular sections entering into the 
scores actually reported for test candidates. These standard deviations indicate that 
neither observed nor estimated score change accurately represents a particular individual 
change in abilities between test administrations. 



SUMMARY AND CONCLUSIONS / 

Despite the fact that the average score change on the Scholastic Aptitude Test is approx- 
imately one-half of the test's standard err<T of measurement, the majority of students 
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electing to repeat the SAT actually experience score changes greater than the standard 
error of measurement. These gains and losses between test administrations largely reflect 
the influence of measurement errors as students with low Initial scores stand to gain 
points upon retesting while students with high initial scores may lose points. Score 
changes on admissions tests such as the SAT, therefore, represent unreliable measures 
of individual growth and development in the abilities tapped by the test. 

A procedure for estimating true score change resulted in a nearly constant estimate 
of score change across levels of initial SAT-Verbal and SAT -Mathematical scores. This 
procedure took the regression effect on score change and possible student self-selection 
in test repetition into account through the use of equating sections, essentially parallel 
and concurrent shorter forms of the SAT-V and SAT-M which do not affect reported scores, 
in obtaining independent estimates of score change between test administrations. The 
success of the procedure in yielding estimates of score change apparently free from the 
effects of measurement error further suggests that the errors of measurement leading to 
the regression effect on score change arise primarily from differences in content, dif- 
ficulty, and other aspects of test specifications for parallel forms and from differences 
in guessing, pacing, omitting, and other components of individual responses across test 
sections. Other procedures, however, would probably be as successful in adjusting for 
these same sources of measurement error (e.g., Lord, 1963). 

The evidence was equivocal regarding possible student self-selection in deciding to 
repeat an admissions test because they perceive their initial test scores as under- 
estimates of their actual abilities. For some samples of test candidates there were the 
expected estimates of overall negative errors of measurement on initial test scores. But 
there was an estimate of an overall positive error of measurement for the initial verbal 
scores of a large number of secondary school juniors later repeating the test as seniors. 
This may simply reflect the error involved in linking the score scales of various equat- 
ing sections or indicate the importance of background variables like high school rank and 
years of study in different subject matters in student perceptions of the consistency of 
their test scores with their academic abilities— factors which the analysis of score 

*ange did not take into account. Lower correlations between verbal and mathematical 
scores among students with repeat test administrations than among students with a single 
test administration also suggest that some students may retake an admissions test because 
they believe differences in scores for separate traits to be a discrepancy rather than an 
indication of their relative strengths. Nevertheless, the analysis of score change re- 
sulted in estimated differences between test administrations more consistent across levels 
of initial test scores and across samples of test candidates than were the observed score 
differences. 

The procedure followed here in estimating true score change depends on scores from 
concurrent and parallel tests on two occasions. It succeeds in correcting for regression 
effects when the correlation between parallel test forms administered at the same time 
(i.e., rx^y and r x 2 Y 2 ^ is nearl y the same as the correlation between separate test 
administrations (i.e., rx^) - Under these conditions the regression effect* noted in 
examining observed score changes by initial levels of test performance apparently 
arises from errors of measurement also found across parallel forms, and the procedure 
accounts for such variations in aspects of test specifications and in responses of test 
candidates. Under other conditions, for example when the correlation between parallel 
forms is much greater than the t^st-retest correlation, the same procedure would probably 
.iot account for all of the sources of variation leading to regression effects. 

Although it has been common practice to relate average score changes on admissions 
tests to their standard error of measurement, such a comparison seems inappropriate when 
change involves two test administrations and when change in the relevant abilities might 
occur over the time between test administrations. Difference scores would then be 
susceptible to errors of measurement on both the initial and a subsequent test administra- 
tion. Indeed, the majority of examinees repeating the SAT experience score changes 
greater than the standard error of measurement. A more appropriate benchmark for 
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judging the meaningf ulness of score change might be the standard error of estimate 
coupled with the intercept constant in predicting subsequent scores from initial scores. 
Regardless of the basis for comparison, the inherent unreliability of difference scores 
suggests that little weight can be given to the observed change in admissions or place- 
ment decisions. 
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APPENDIX: Observed and Estimated Score Changes from Junior to Senior Year 



Verbal Score Change Mathematical Score Change 

Junior Observed Estimated Observed Estimated 

Year Mean Mean Mean Mean 

Score N Change Change N Change Change 
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APPENDIX: Observed and Estimated Score Changes from Junior to Senior Year (continued) 



Verbal Score Change Mathematical Sc ore Change 

Junior Observed Estimated Observed Estimated 

Year Mean Mean Mean Mean 

Score N Chan 8 e Change N Change Change 
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